Skip to content

[None][infra] allow to choose repo when generate lock files #8659

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[None][infra] allow to choose repo when generate lock files #8659

wants to merge 5 commits into from

Conversation

@yuanjingx87
Copy link
Collaborator

@yuanjingx87 yuanjingx87 commented Oct 24, 2025 •
edited by chzblych
Loading

Summary by CodeRabbit

  • New Features

    • Extended pipeline capabilities to support dynamic repository selection, including custom repositories, and configurable branch parameters for lock file generation workflows

  • Chores

    • Updated build environment configuration to properly exclude security-related file patterns from code quality checks

Description

Allow user to generate lock files for gitlab, or a customized repo.
Skip codespell check in precommit hook for lock files and pyproject.toml files

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@yuanjingx87 yuanjingx87 requested review from a team as code owners October 24, 2025 19:08
@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch from 4a0bdee to 5c032e7 Compare October 24, 2025 19:10
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 24, 2025 •
edited
Loading

📝 Walkthrough

Walkthrough

Pre-commit configuration adds an ignore pattern for security_scanning/* files to the codespell hook. Jenkins pipeline refactors parameter handling to support dynamic repository URL selection with fallback for custom repositories and renames branch parameter from llmBranch to branchName across all Git operations.

Changes

Cohort / File(s) Summary
Pre-commit configuration
.pre-commit-config.yaml		 Extended codespell hook arguments with additional ignore pattern for security_scanning/* files
Jenkins pipeline parameters and operations
jenkins/GenerateLock.groovy		 Updated pipeline parameters: renamed llmBranch to branchName, added repoUrl choice parameter with predefined options and customRepoUrl for custom repositories. Modified LLM_REPO initialization to use params.repoUrl with conditional fallback to params.customRepoUrl. Updated all Git operations (fetch, rebase, push) to reference params.branchName

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Description Check ⚠️ Warning The pull request description has significant gaps compared to the required template. The Description section is present but quite brief and contains a potential discrepancy—it states the changes will "skip codespell check in precommit hook for lock files and pyproject.toml files," yet the raw_summary shows the actual change adds an ignore pattern for security_scanning/* files instead. More critically, the Test Coverage section is completely empty, containing only a comment placeholder with no actual test cases listed, which is a required section per the template. The PR Checklist is present but consists largely of boilerplate template text without specific customization to indicate what items were actually verified for this particular change. The author should expand the Description section to accurately reflect the actual changes made, including the specific ignore patterns added to the codespell hook configuration and the dynamic repository selection changes in GenerateLock.groovy. The Test Coverage section must be completed with a clear list of relevant tests that validate these changes. Additionally, the author should review and customize the PR Checklist items to confirm which guidelines and requirements were actually addressed, rather than leaving the template boilerplate intact.
✅ Passed checks (1 passed)
Check name Status Explanation
Title Check ✅ Passed The PR title "[None][infra] allow to choose repo when generate lock files" clearly refers to the primary change in the GenerateLock.groovy file, which adds the ability for users to dynamically select a repository when generating lock files through new pipeline parameters. While the PR also includes changes to .pre-commit-config.yaml to skip codespell checks, the title appropriately focuses on the main objective and primary change. The title is concise, specific, and would help teammates quickly understand the core purpose of the changeset when scanning history.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch from 5c032e7 to dd4c427 Compare October 28, 2025 19:46
@yuanjingx87 yuanjingx87 changed the title [None][infra] User/yuanjingx/choose repo when generate lock files [None][infra] allow to choose repo when generate lock files Oct 28, 2025
@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch 4 times, most recently from 7ef2d74 to 90b0000 Compare October 28, 2025 23:14
@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch from 90b0000 to 8316779 Compare October 28, 2025 23:17
chzblych
chzblych approved these changes Oct 29, 2025
View reviewed changes
jenkins/GenerateLock.groovy Outdated
if (repoUrl ==~ /https:\/\/github\.com\/[\w\-]+\/[\w\-\.]+(\.git)?/) {
return 'github-token-trtllm-ci'
}
if (repoUrl ==~ /https:\/\/gitlab\-master\.nvidia\.com\/[\w\-]+\/[\w\-\.]+(\.git)?/) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we were asked to avoid exposing the internal repo URL. There is a Jenkins credential for it, i.e., default-llm-repo or default-git-url.

@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch from 979342e to 9c99663 Compare October 29, 2025 19:48
@yuanjingx87
Hide internal repo url
fcfba9c 
Signed-off-by: Yuanjing Xue <[email protected]>
@yuanjingx87 yuanjingx87 force-pushed the user/yuanjingx/choose_repo_when_generate_lock_files branch from 9c99663 to fcfba9c Compare October 29, 2025 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zeroepoch zeroepoch zeroepoch approved these changes

@chzblych chzblych chzblych approved these changes

@ruodil ruodil Awaiting requested review from ruodil ruodil is a code owner automatically assigned from NVIDIA/trt-llm-ci-infra-devs

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yuanjingx87 @zeroepoch @chzblych